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EDITING  KKX)DS4 


INTRODUCTICN 

This  project,  funded  by  the  Naval  Ocean  Research  and  Development  Activity 
(NORDA)  in  July  1987,  allowed  Compass  Systems,  Inc.  (CSI)  to  begin  editing  the 
data  in  the  ^Jaster  Oceanographic  Observation  Data  Sat  (lyJOODS).  The  present 
version  of  MOODS,  M0QDS4,  was  produced  by  the  third  update  of  MOODS  that  CSI 
ccnplated  ac  Fleet  Numerical  Oceanography  Center  (FNOC),  Monterey,  CA  in 
September  1985.  The  MCODS4  tapes  contain  4.6  million  reports  including  the 
National  Oceanographic  Data  Center  (NODC)  physical  oceanographic  profile  data 
holdings,  all  of  the  real  time  subsurface  reports  that  have  been  received  at 
FNOC  and  other  profile  data  obtained  from  a  variety  of  sources,  including  re¬ 
search  institutions  and  foreign  navies.  The  reports  in  MOODS  had  been  passed 
through  a  primitive  set  of  quality  tests  that  had  set  flags  but  had  not  re¬ 
moved  observations.  All  previous  efforts  to  create  and  update  NWODS  were 
directed  toward  aggregating  the  data  frcm  multiple  sources  with  )cnowledge  that 
seme  but  not  all  of  the  data  in  sente  of  the  newer  sets  duplicated  reports  al¬ 
ready  included.  This  project  started  the  process  of  ranoving  these  redundant 
data  and  took  advantage  of  the  improvements  made  in  processing  real  time  data. 

After  the  M0CDS4  tapes  were  completed  at  FNOC  the  Naval  Oceanographic  Of¬ 
fice  (NAVOCEANO)  assumed  responsibility  for  maintaining  >KODS.  The  results  of 
this  editing  contract  will  not  appear  in  the  MOODS  file  until  NAVOCEANO  produces 
a  IOCDS5  set. 

Beginning  in  June  1985  CSI  was  funded  by  the  U.S.  Tropical  Ocean  Global  At¬ 
mosphere  (TOGA)  project  office  of  NOAA  (1)  to  access  the  real  tLme  data  and  to 


produce  monthly  quality  controlled  extracts  of  these  data.  The  TOGA  project 
resulted  in  the  installation  of  the  Zenith  Z-100  based  microcomputer  system 
labelled  the  Quality  Iinprovement  Profile  System  (QUIPS)  that  has  been  used  at 
FN(X  since  toril  1986  to  edit  both  unclassified  and  classified  reports.  Start¬ 
ing  in  January  1986  CSI  began  editing  the  unclassified  reports  extracted  from 
the  FNOC  message  files  to  remove  non-exact  duplicates,  correct  call  signs  and 
reccmbine  tatperat’ure  and  salinity  profiles.  This  mini  MOODS  update  procedure 
split  the  FNOC  MOODS  310  file  archive  tape  into  files  of  classified  and  'un¬ 
classified  MOODS  data  and  created  a  monthly  update  to  the  master  MXOS  tapes  at 
FNOC.  The  level  of  non-real  time  quality  control  (QC)  increased  throughout 
1986,  as  the  National  Ocean  Service  (NOS)  and  the  FN(X-resident  NODC  enroloyee 
assumed  responsibility  for  producing  weelcly  sets  of  edited  subsurface  reports  in 
Septanber,  ending  CSI's  funding  for  maintaining  the  TCGA-related  monthly  .^iOODS 
files.  In  October  1986  NAVOVCEANO  and  FNOC  chose  not  to  continue  using  the 
CSI-developed  procedures  for  maintaining  the  MOODS  update  stack  tapes  in  the 
FNOC  operational  center  or  to  utilize  the  edited  version  of  the  unclassified 
data  for  operational  extracts  (2). 

The  minutes  of  the  MOODS  workshop  held  at  NORDA  17-18  October  1985  (3)  con¬ 
tain  a  list  of  recommendations  including  the  creation  of  an  edited  version  of 
MOODS,  which  was  estimated  would  cost  between  $100,000  and  $150,000.  This 
$75,000  project  was  aimed  at  producing  edited  subsets  of  the  MOODS4  data  that 
would:  1)  correct  known  conversion  errors  and  remove  redundant  data  in  the 
hydrocast  source  sets;  2)  calibrate  bathythermograph  reports  so  that  they  could 
be  combined  with  other  data  in  statistical  tests;  and  3)  make  an  initial  edit 
pass  of  the  message  data,  applying  the  TCGA-developed  correction  techniques. 


The  description  of  these  three  edit  tasks  frcm  the  CSI  proposal  (4)  follows: 

Task  1,  Replace  NOEXT  Hydrocast  Records. 

1.  Obtain  and  convert  the  entire  NOEC  Universal  Station  II  (USD2)  file. 

2.  Extract,  resort  into  platform  order  and  rerun  the  speed  test  on  the 
following  MOODS  source  cedes  to  identify  redundant  data  sets:  17,  18, 

19,  21,  22,  26,  27  and  30.  (See  extract  of  TABLE  1  from  the  MOODS 
Functional  Description  (5)  included  before  References.) 

3.  Create  sorted  speed  record  files  during  the  conversion  and  check  the 
above  sets  for  any  additional  redundancies. 

4.  Eliminate  fron  MOODS  the  old  source  code  4  and  15  sets  and  any  others 
that  are  redundant,  including  recycling  the  appropriate  backup  and 
source  tapes  in  the  MOODS  archive. 

Task  2.  Calibrate  NOEXI  Northern  Hgnisphere  Mechanical  Bathythermograph  Data. 

1.  Extract  the  source  code  3  mechanical  bathythermograph  (MET)  data  frcm 
M00DS4  and  then  extract  data  in  the  northern  hanisphere  cruises  in  the 
NODC  processed  MBT  reference  number  range  that  have  reference  tempera¬ 
tures  but  not  temperature  correction  (TCS)  values. 

2.  Resort  these  data  by  reference  number,  instrument  code  and  consecutive 
number.  Calculate  a  TCS  for  each  instrument  and  plot  the  data  by  in¬ 
strument  for  review. 

3.  Review  the  calculated  TCS  values  greater  than  .S^C  using  the  listing  and 
plots  and  prepare  a  correction  file. 

4.  Apply  the  corrections  and  submit  the  combined  set  of  northern  and  pre¬ 
viously  calculated  southern  henisphere  corrections  to  NODC. 


1.  Extract  message  data  frcm  the  >C0DS4  tapes  into  a  separate  file  and 
divide  the  extracted  data  into  3  subsets:  surface  only  reports, 
hydrocast  ccmponen*-.  reports  and  temperature  only  reports. 

2.  Test  data  failing  temperature  range  tests  and  attempt  to  correct, 
assuming  they  were  converted  from  Centigrade  rather  than  Fahrenheit, 
using  both  the  surface  and  120M  temperature  climatologies.  Attenpt 
to  correct  reports  with  excessive  depths  by  'using  120M  temperatures 
and  by  assuming  that  the  depths  are  in  feet  rather  than  10 's  of  feet. 
Check  for  increasing  depths  and  set  a  profile  flag  for  reports  failing 
the  IGOSS  spike  test  (6). 

3.  Resort  the  data  into  cruise  order  and  revise  the  ship  call  signs  using 
the  new  updates  to  the  FNCC  ship  code  file. 

4.  Using  software  developed  for  TOGA  combine  the  hydrocast  cctrponent 
reports  into  a  single  J400DS  record,  removing  nonexact  duplicates  in 
the  process. 

5.  Using  TOGA  software,  eliminate  nonexact  duplicates  which  are  at  the 
same  time  and  position  but  have  different  numbers  of  levels. 

6.  Merge  the  subsets  and  reset  the  speed  flags  using  the  improved  list  of 
collective  call  signs. 

DISCUSSION 

The  tasks  selected  were  those  which  CSI  believed  would  provide  the  greatest 
benefit  for  the  available  funds  and  were  tied  directly  to  NORDA's  Lmrr,ediate 
needs.  Task  1  reprocessed  and  eliminated  duplicates  within  the  hydrocast  source 
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files,  which  provide  all  of  the  matched  tenperature  and  salinity  values  for  the 
calculation  of  sound  velocity  profiles  and  a  few  directly  measured  sound  veloci¬ 
ties.  Task  3  conbined  the  real  time  tenperature  and  salinity  profiles  so  that 
they  would  also  be  accessible  for  so'ond  velocity  calculations.  'Tasks  2  and  3 
eliminated  or  flagged  reports  that  create  nrajor  biases  in  statistics  calc’Jlated 
from  the  YjOCDS  data  by  calibrating  the  MBT  reports  and  performing  a  whole  series 
of  correction,  flagging  and  duplicate  elimination  operations  on  the  real  time 
data.  Each  of  the  task  elements  is  discussed  belcw  in  sequence. 

Task  1. 

NCCC,  having  undergone  a  number  of  computer  hardware  chengeovers,  has  final¬ 
ly  arrived  at  a  stable  configuration  on  the  UNIVAC  coputer  at  Asheville.  While 
CSI  had  obtained  and  incorporated  the  latest  updates  from  XCDC  for  each  of  the 
3  update  cycles,  the  data  tapes  had  been  in  different  formats  from  different 
NODC  ccnputer  systems.  There  was  evidence  that  seme  North  Pacific  data  had  been 
lost  and  there  were  zero  salinity  values  in  the  file  that  were  not  marked  as 
missing  values.  Additionally,  NODC  had  trade  a  significant  effort  to  remove 
duplicate  cruises  and  correct  data  in  this  fils.  Rather  than  duplicate  their 
editing  effort,  CSI  arranged  with  NODC  to  obtain  a  new  copy  of  the  entire  file 
at  no  cost  to  the  Navy.  The  NCODS  conversion  routine  was  modified  to  insure 
that  all  converted  salinity  values  were  within  a  reasonable  range,  and  the  re¬ 
sults  of  the  conversion  and  >iOCDS  error  flag  settings  summarized  by  NODC  cruise 
nun±er  were  sent  back  to  N(XC. 

The  forward  processing  direction  dictated  by  the  past  MXDS  updates  produced 
a  sat  of  edit  flags  in  each  source  set.  The  conversion  and  processing  sequences 
were  intended  to  flag  duplicate  reports  of  lesser  quality;  however,  there  had 
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never  been  time  to  produce  a  listing  that  showed  the  number  of  interfile  dup¬ 
licates  flagged  in  each  source,  and  within  each  source  the  number  of  flags 
of  each  type  sat  within  each  cruise.  This  second  alanent  of  Task  1  modified 
the  existing  codes  in  the  speed  test  sequence  to  summarize  the  flags  set.  The 
last  element  in  the  task  was  to  eliminate  and  recycle  the  old  hydrocast  source 
tapes  in  the  >!CODS  archive  at  FNOC,  thereby  releasing  approximately  140  tapes. 
Task  2 • 

The  second  task  corrected  a  long-standing  problem  with  the  the  NODC  MBT 
file.  When  NCDC  digitized  MBT  observations  they  did  not  calculate  .and  apply 
instrument  corrections;  ccnsaquencly,  even  though  the  reference  temperature  may 
have  been  offset  from  the  profile  surface  tsnperature  by  a  significant  constant 
temperature  difference,  the  profile  tarperatures  were  never  adjusted.  This  was 
a  common  problem  with  the  MET’s  and  was  caused  by  the  bending  of  the  arm  holding 
the  stylus  in  the  instrument  at  any  time  after  the  calibration  grid  was  made. 
^J0DC  considered  the  MET  data  to  be  only  a  relative  measurement,  but  in  practice 
most  researchers  who  use  >sr  data  treat  Lhem  as  if  they  were  calibrated.  WTcen 
CSI  used  MBr  data  to  produce  monthly  temperature  atlases  for  the  northern  hemi¬ 
sphere  the  offsets  were  handled  by  computing  average  difference  means  (7),  i.e., 
creating  an  average  profile  by  adding  the  average  differance  between  standard 
levels  to  the  surface  mean  value.  While  this  produced  usable  inputs  to  a  labor- 
intensive  edit  process,  it  is  not  recommended  for  purely  statistical  analyses 
since  it  still  relies  on  the  validity  of  the  surrace  mean  loncwn  to  contain 
biased  values.  In  the  Southern  Hanisphere  Atlas  project  CSI  attempted  to  use 
the  .'-IST  data  without  calibration  and  discovered  it  was  impossible  because  the 
sample  was  so  small  that  the  biases  were  obvious.  CSI  subsequently  calculated 


corrections  for  all  MET  cruise  sets  in  the  southern  hemisphere  and  applied 
these  corrections  to  the  MET  data  in  ^CODS.  The  N’ODC  MBT  file  is  now  part  of 
their  UBT  set.  WTien  NODC  converted  their  MB!  file  to  UBT  format,  the  essential 
instrument  code  needed  to  separate  the  reporrs  by  instrument  was  discarded, 
so  that  NCCC  could  no  longer  correct  the  MBT  reports,  .as  CSI  had  already  com¬ 
pleted  the  corrections  for  the  southern  hemisphere,  the  software  to  carry  out 
this  task  was  available  and  tested  on  the  FN'CC  computers.  This  was  a  labor- 
intensive,  one-time  operation,  but  it  had  to  be  done  if  the  MBT  reports  were  to 
continue  to  be  included  in  the  samples  on  which  variability  statistics  are 
calculated.  Since  the  MBT's  represent  over  a  third  of  the  unique  observations 
in  MCCDS  and  represent  the  older  data,  CSI  believed  .'IBr  data  were  Lmportant. 

To  ccnplete  this  task  the  MBT  source  file  was  extracted  from  the  MCCDS4 
file  and  divided  into  two  parts.  The  reports  in  the  cmaise  number  ranges  not 
assigned  to  the  Scripps  Institution  of  Oceanography  (SIO)  were  separated  from 
the  other  reports  and  passed  through  the  correction  sequence  if  the  cruise  had 
referance  temperatures  and  had  observations  without  previous  temperature  cor¬ 
rections.  These  cruises  were  reviewed  for  consistency  and  trends  by  looking  at 
the  surface  temperature,  the  calculated  difference  fron  the  reference  tem¬ 
perature  and  the  Bauer-Robinson  climatology  values  at  the  surface  and  120 
meters.  Reports  that  showed  offsets  indicating  the  slide  ''.vas  not  properly 
registered  in  the  instrument  mount  were  adjusted  with  special  corrections  or 
deleted.  The  corrections  were  applied  to  the  MOODS  file.  A  special  correction 
file  was  then  prepared,  providing  the  cruise  number,  observation  number,  time, 
position  and  TCS  correction,  so  NOCC  could  correct  its  uBT  file. 


Task  3. 


The  real  time  data  had  never  been  edited  and  were  pieced  together  from  many 
undoc'cmented  tapes  found  in  the  FNCC  tape  locker.  Most  of  the  early  reports 
were  at  one  time  on  cards  that  scmetimes  got  out  of  order  as  they  were  handl^ed, 
resulting  in  garbled  reports.  Based  on  CSI's  study  of  these  data,  many  were 
unique  reports  not  in  the  processed  NOIX:  files,  making  it  potentially  a  very 
valuable  fils.  However,  its  enormous  size — over  one  million  reports — and  its 
poor  quality  made  it  the  mosr  difficult  to  handle.  .Additionally,  the  edit  flag 
rests  which  were  developed  primarily  for  cruise  order  processing  the  XBT  data 
were  poorly  suited  to  this  data  set.  Specifically  Ine  >!CCDS  default  30  .<not 
speed  test  was  applied  to  reports  made  by  aircraft  and  to  reports  mads  on  dif¬ 
ferent  platforms  bur  identified  by  a  collective  platform  code.  The  intrafile 
nonexact  duplicate  flag  test  does  not  distinguish  between  a  hydrccast  and  3T 
temperat’ore  profile  and  looks  only  at  the  number  of  levels  rather  than  the 
ma^<imum  depth,  so  nonexact  duplicate  flags  were  set  on  reports  that  were  unique. 
Nonexact  duplicate  reports  are  those  that  occur  at  the  same  time  and  location 
bur  have  differant  profile  data. 

To  edit  the  real  time  fils,  it  was  divided  into  three  segments:  one  contain¬ 
ing  s’orface  only  reports,  which  are  usually  made  by  aircraft;  the  second  con¬ 
taining  hydrocasr  component  reports  and  the  third  containing  only  temperature 
profile  m.eas’urement3.  The  hydrccast  reports  normally  are  divided  into  two 
separate  reports  at  FTIOC:  one  containing  tem.perat’ares  and  the  second  containing 
salinities.  Both  parameters  are  originally  reported  at  the  same  depth  levels, 
but  the  number  of  levels  may  be  truncated  in  either  ccmponent  and  intermediate 
levels  are  lost  when  the  data  are  converted  to  MCCDS  if  they  can  be  reccrrputed 
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using  linear  interpolation.  Using  codes  developed  for  processing  these  data  for 
lUGA,  the  corronents  were  reconbined  into  a  single  report  concaining  both  tem¬ 
perature  and  salinity  with  the  missing  data  values  included  if  the  levels  re¬ 
ported  did  not  match.  .All  of  the  real  time  profiles  were  then  edited  using 
codes  that  were  developed  for  TOGA. 

CHRa^OLOGY 

The  original  project  schedule  was  specified  so  that  the  results  from  this 
contract  effort  would  feed  into  the  installation  of  >50CDS  on  the  NAVOCEAi^O  con- 
puter  during  the  3uirrr.er  of  1987.  The  project,  however,  had  two  links  to  the 
TOGA  grant  effort:  CSI's  staff  was  working  on  both  projects  simultaneously  and 
the  planned  use  of  the  TOGA-developed  editing  techniques  on  the  historical  >500DS 
real  time  data  source  set  in  Task  3.  After  the  initial  start  on  the  edit  tasks, 
which  included  extracting  the  MBT  and  radio  message  source  sets  from  >03034  and 
making  sure  that  the  MASTER  program  could  read  and  convert  test  tapes  of  USD2 
tapes  produced  by  .NODC  on  the  Asheville  computers,  most  of  CSI’s  efforts  were 
devoted  to  the  TCGA  programs.  Between  July  and  September  the  TCGA  effort  con¬ 
sumed  all  of  CSI's  resources  as  the  weekly  near -real  time  message  edit  sequence 
replaced  the  monthly  TOGA  processing  and  the  access  through  NODDS  replaced  the 
monthly  mini  >03DS  update  cycle.  CSI  developed  the  mini  >XX!DS  update  cycle  and 
ran  it  monthly  from  Febriuary  through  September  1987,  maintaining  an  updated 
version  of  the  MOODS  classified  data  tape  and  an  unclassified  >50003  stack  tape 
in  the  FN'OC  operational  center.  After  the  first  runs  of  the  mini  >500DS  update 
in  February  1987  resulted  in  a  recall  of  the  downgraded  data,  the  downgrading 
feature  was  inhibited,  but  the  mini  >S00DS  update  cycle  still  served  to  separate 


the  classified  and  'unclassified  data,  set  >SCX)DS  QC  flags  and  mintain  current 
MOCDS  tapes  in  the  FN(X  center.  Vvhen  the  OAG  near-real  tiire  edit  cycle  began 
functioning  in  Septsmber,  unclassified  data  had  to  be  extracted  weekly  rather 
than  monthly.  FNCC,  having  decided  not  to  use  the  near  real-time  edited  data 
operationally,  also  elected  not  to  continue  the  mini  .\CCDS  update  procedure,  so 
che  operational  MCCDS  tapes  have  reverted  to  the  MCCDS4  sac. 

By  December  the  MCDDS  (QUIPS)  process  '/jas  running  smoothly,  and  time  was 
available  to  extract  the  different  source  sets  from  MXDDS  to  see  how  much  re¬ 
dundancy  there  'was  in  the  different  hydrccast  and  X3T  .data  sets.  Table  1  sum¬ 
marizes  the  results  and  shows  that  none  of  the  source  sets  'were  completely  re¬ 
dundant.  In  December  CSI  'was  also  able  to  run  some  TCS  plots  using  the  program 
developed  to  correct  the  MBT  reports  for  our  Southern  Hemisphere  Atlas.  After 
working  'with  these  listings  for  several  'wee^'cs,  it  became  clear  that  excessive 
time  was  req'uired  to  look  up  values  to  determine  if  reference  temperatures  ware 
bad  and  that  a  better  job  could  be  done  in  less  time  -JLf  the  Bauer-Robinson 
Numerical  Atlas  values  were  shown  on  the  listing.  Consequently,  the  edit  cor¬ 
rection  efforts  ware  suspended  until  the  program  could  be  modified. 

In  February  1987  Mr.  Bauer  visited  NAV(XEANO  and  saw  that  their  inplementa- 
tion  schedule  'was  slipping.  Also,  on  that  trip  CSI's  TCGA  efforts  ware  given 
an  increased  priority  'when  the  decision  was  made  to  update  the  QUIPS  capacity  at 
the  National  iMeteorolcgical  Center  (NMC)  to  use  the  microVAX  workstation  for 
real  tune  QC  of  all  marine  data  'with  a  'watch  staff  that  had  to  be  hired  in  FY 
1987.  This  addition  to  the  TOGA  grant  effort  consumed  the  major  portion  of  Mr. 
Bauer's  tune  throughout  the  remainder  of  the  project  and  resulted  in  the  req’uest 
for  extension  of  the  contract  performance  period  to  1  November  1987. 


The  final  shipment  of  the  last  10  tapes  containing  the  NOCX!  uSD2  file  was 
received  in  mid  February  and  the  full  set  of  19  tapes  was  converted  to  I'-JOCDS 
during  March  and  early  April.  For  the  conversion  of  z'r.e  USD2  file,  the  MjXDLGO 
program  ac  FNOC  was  modified  to  check  for  zero  salinity  and  to  change  the  vari¬ 
able  header  to  include  waner  color  and  transparancy  (MCCDLGO  update  40).  NKX)OS4 
quality  tests  were  run  on  the  set,  and  the  MGEO  order  tape  for  source  code  4 
was  delivered  to  MCRDA  on  24  April  1987.  This  tape  contained  686,752  reoorts. 

The  conversion  run  listing  shewing  decode  errors  and  zero  salinity  locations 
was  given  to  »CDC  on  4  June  1987.  Additionally,  the  final  tape  was  resorted 
into  cruise  order  and  the  speed  set  printout  was  modified  in  M£CDLGO  to  show 
how  many  >XXDS4  flags  were  set  in  each  cruise.  NCDC  has  used  this  lisning  to 
extract  the  cruises  with  decode  and  duplicate  errors  and  is  now  wording  to 
revie*//  other  cruises  with  high  error  flag  counts. 

Additional  runs  were  made  to  determine  if  any  of  the  M0CDS4  source  sets 
could  'oe  eliminated  based  on  the  contents  of  the  USD2  update.  The  results 
showed  that  none  could  be,  but  NCIX:  requested  and  was  givan  copies  of  the  Japan¬ 
ese  hydrocast  tape.  CSI  also  provided  copies  of  the  axtracted  reports  from  the 
FNOC  CALMA  XET  digitization,  which  have  now  been  converted  to  UBT  format  by  XCEX: 
and  are  being  screened  for  inclusion  in  the  NCDC  U3T  file. 

Cn  1  May  1987  CSI  shipped  NCRDA  the  unclassified  rressage  data  edited  as  part 
of  the  TOGA  project  for  1985  and  1986.  These  tapes,  which  were  also  sent  to 
NCDC,  are  the  "best"  global  versions  of  the  FNOC  unclassified  message  data,  al¬ 
though  the  NCDC  TOGA  report  file  is  better  for  the  Pacific  TOGA  area.  The  1985 
data  combined  reports  extracted  fraxi  MCODS4  and  the  data  axtracted  from  the 
FNOC  MOODS  310  archive  classified  tape  without  any  downgrading.  In  com,parison 
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with  the  1986  tape,  the  1985  data  set  is  only  partially  edited.  Sate  non  exact 
duplicates  v;are  reiroved,  salinities  were  fixed  and  the  call  signs  were  passed 
though  the  garble  table.  The  1986  tape  shews  the  results  of  increasing  levels 
of  non-real  tuTe  editing.  The  data  before  Septeenber  were  only  edited  on  the 
monthly  schedule  with  no  graphic  interactions.  Starting  in  September  1987, 
call  signs  and  profiles  have  been  edited  with  the  aid  of  time  series  plots, 
geographic  plots  and  the  near-real  time  QUIPS  station.  These  efforts  are  de¬ 
scribed  in  CSI's  TOGA  report  (8). 

After  the  delivery  of  these  two  tapes,  CSI  began  editing  the  historical  pre- 
1985  data.  The  major  portion  of  the  effort  was  devoted  to  modification  of  the 
call  sign  using  the  concept  of  the  garble  table.  Hundreds  of  runs  were  made  to 
identify  safe  call  sign  translations  for  inclusion  in  the  garble  table.  The 
MC0DS4  message  edit  processing  section  of  this  report  provides  a  detailed  docu- 
m.entation  of  this  major  effort. 

In  July  1987  the  complete  set  of  MET  plots  and  listings  was  created  with  the 
additional  Bauer-Robinson  Numerical  Atlas  differences  at  0  and  120  meters.  The 
hand  editing  of  these  data  was  done  entirely  by  Mrs.  Margaret  Robinson,  who  had 
headed  the  Scripps  Institution  of  Oceanography  Bathythermograph  Processing  Sec¬ 
tion  while  the  instrument  was  in  active  use.  The  details  of  this  process  are 
documented  in  the  MBT  Correction  Process  section  of  this  report. 

The  two  real  tLme  update  tapes,  the  pre-1985  ’unclassified  data  and  the 
M00DS4  classified  data,  were  shipped  to  NORDA  by  FNOC  on  20  October  1987.  The 
final  MBT  update  was  shipped  to  NORDA  by  CSI  on  29  October  1987. 


The  following  sections  cn  the  MBT  correction  process  .and  the  real  time  adit 
process  contain  detailed  descriptions  of  how  the  corrections  were  accomplished. 


and  are  included  prLTarily  as  an  aid  if  these  efforts  are  ever  continued  in  the 
future.  The  casual  reader  rray  wish  to  skip  these  sections. 

MBT  C0RR5CTICN  PROCESSING 

The  TCS  correction  applied  to  the  ^lET  data  is  a  statistic  correction  based 
on  the  assurnption  that  the  reference  tafnperature  is  absolute  and  the  profile 
measured  by  the  MBT  is  relative.  The  correction  is  ccmputed  separately  for 
each  instrument  used  on  a  cruise  and  on  occasion  may  be  computed  for  subsets  of 
chose  observations  when  a  shift  in  calibration  is  observed.  The  procedure  is 
sLiple  but  necessary  if  the  tsmoerature  maasurement  made  with  an  MBT  is  to  be 
averaged  with  ^OT's  or  hydrocast  data  or  is  to  be  used  in  determining  a  distri¬ 
bution  of  temperatures  for  an  area. 

The  processing  sequence  is  outlined  in  Table  2.  To  calculate  the  TCS  cor¬ 
rection  all  MBP  reports  with  instrument  codes,  sources  3  and  9,  were  extracted 
from  the  M0CDS4  tapes — a  total  of  1,039,030  observations.  This  file  was  then 
divided  by  program  MBTSPLT  into  two  files.  The  first  contai.ied  347,667  reports 
that  were  from  Australia,  had  no  instrument  codes,  or  were  in  the  N’CCXP  cruise 
number  block  10,000  to  15,000,  which  had  been  processed  at  310  and  had  already 
had  TCS  corrections  applied.  The  second  file  had  691,363  reports,  which  were 
then  sorted  into  cruise,  instrument  and  time  order.  Program  MBTOrA)  split  this 
second  file  into  a  file  of  131,515  correctable  reports  and  a  file  of  559,847 
reports  that  had  already  'oeen  corrected  or  were  uncorrectable .  The  correctable 
reports  were  listed  and  plotted  with  a  modified  version  of  the  CHKTCS  program 
originally  created  to  edit  southern  hemisphere  data  before  a  numerical  atlas 
was  available. 
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TABLE  2 


TCS  SEQUENCE 


JOB  JOB 

STEP  PROGRAM 

INPUT 

FILE 

OUTPUT 

FILE 

FUNCTION 

1 .  MOODEXT 

^O^DS4 

69152 

Extract  sources  3  and  9 

2 .  MBTSPLT 

69152 

69720 

MBTNOCOR 

First  separation  of  MBT  reports 
with  blank  Inst  Code,  10,000- 
16,000  cruise  numbers  and  Aus¬ 
tralian  MBT's 

3 .  MBTSORT 

59270 

MBT( 69723) 
69724 

Sort  by  Inst,  NCEX)  cruise  and 
time 

4.  CHKTCS 

MB! 

TSCARDn 

Compute  and  plot  'TCS  correc¬ 
tions 

5 .  CCNSECSRT 

MBT 

MBTCCN 

Sort  by  Inst,  NODC  cruise, 

NODC  consecutive  number 

6 .  MODFTCS 

MHTCON 

MBTTCSCOR 

Apply  TCS  corrections 

7 .  CCNSRT 

MBTTCSCOR 

METSRT 

Sort  by  NODC  cruise  and  con¬ 
secutive  number 

8 .  MBTDEL 

MBISRT 

MBTNEW 

Delete  reports 

9 .  GECSRTQU.aLL 

SRIMOOD 

MBI'NEW 

METGEOSRT 

MGEO  sort 

10 .  QUAL 

MOODLGO 

MBT3EOSRT 

MBINEMGEO 

MOODS  MGEO  flag  set 

11 .  iMBTMERGS 

SR•I^KX}D 

MBTNEWGEO 

69720 

69724 

32487 

MGEO  merge 

12 .  DUPS'rOT 

MOODIGO 

32487 

18151 

Eliminate  duos  and  reset  MGEO 
MOODS  flags  ' 

13 .  TOrALMHT 

18151 

Check  final  tape 

Note:  All 

jobs  on  ULIB  fi 

le  UMBTPGM,ID= 

=R3 

The  irodif ication  added  a  ccnparison  of  the  MET  ceoperatures  and  the  Bauer- 
Robinson  Numerical  Atlas  at  the  surface  and  120  meters.  Mditionally,  the 
program  was  changed  to  plot  all  reports  rather  chan  just  those  where  the  com¬ 
puted  TCS  value  exceeded  .5°C.  'The  addition  of  the  atlas  values  greatly  speeded 
the  subjective  aval-uation  and  provided  additional  clues  to  identify  cut-cf- 
position  reports  and  cases  where  the  MET  glass  slide  was  improperly  registered 
in  the  instrument  or  when  it  had  been  photographed. 

The  subjective  edit  process  took  approximately  six  man  wee;<s  spread  over  a 
period  of  three  months.  It  resulted  in  a  file  of  TCS  corrections  that  could  be 
applied  to  all  reports  made  by  the  instrument  in  the  cruise,  a  file  of  special 
corrections,  a  file  of  deletes  and  a  list  of  cruises  that  NDCC  needs  to  check 
for  positions. 

To  apply  the  corrections,  the  correctable  file  had  to  be  resorted  by  NCDC 
cruise  number,  instrument  number  and  consecutive  number.  The  96,863  correc¬ 
tions  were  applied  by  program  MCDFTCS  and  then  the  fils  was  resorted  into 
cruise  number,  consecutive  n'umber  order  to  ma:<e  the  387  deletes. 

The  edited  file  of  131,129  reports  was  resequenced  and  passed  through  the 
standard  MGEO  sequence  edit  flag  setting  procedure  using  the  MOODIO  procedure. 
This  procedure  eliminated  7  exact  duplicates,  found  19  interfile  duplicates, 
set  1112  land  flags  (mainly  over  the  Great  Lalos)  and  588  SST  flags  (with  a 
large  number  ’oeing  in  cruises  ma  ie  cut  of  Norfor<,  where  the  positions  w’ere 
suspected  in  the  TCS  listing.)  The  correct  file  was  rtterged  with  the  two 
uncorrected  or  uncorrectable  report  files  generated  by  the  MBTSPLT  and  MBTDIVD 
programs  to  produce  a  new  )'5GEO  order  file.  Because  of  evide.nca  that  there  were 
seme  exact  duplicates  in  the  original  .'■)OODS4  file  caused  by  the  incomplete  sort 


specification  used  in  early  MD3DS  updates,  the  entire  file  was  subjected  to  a 
resetting  of  the  >GEXD  MXDS  flags  using  procedure  J100DI0.  This  process  elim¬ 
inated  an  additional  27  reports  and  identified  4,743  non  exact  duplicates  prob¬ 
ably  introduced  by  the  Australian  MET  updates.  There  were  1,038,609  reports  on 
the  >KX)DS  update  tape  and  2,865  have  land  flags  set  (again,  primarily  in  the 
Great  Lakes,  which  are  blocked  out  of  the  land  sea  table)  and  5,584  SST  flags. 
There  were  only  4  reports  with  maximum  depth  errors.  The  list  and  tape  of 
corrections  and  deletes  was  sent  to  MCDC  so  they  could  update  their  (JBT  file. 

MOCDS4  MESSAGE  EDIT  PR0CESSI>X3 

The  edir  process  for  the  MCCCS  source  2  was  divided  into  a  sequence  of  jobs, 
each  composed  of  a  series  of  procedures  and  prograia  steps.  Standardized  prog¬ 
ram,  procedure  and  intermediate  data  file  naming  conventions  were  established 
to  provide  well-defined  interfaces  between  steps  and  jobs  and  to  facilita  re¬ 
starts  when  they  were  necessary.  Each  developed  entity  was  tested  using  several 
data  subsets.  Each  job  was  described  as  a  job  set  as  each  was  applied  to  the 
several  subsets  of  the  data  defined  during  the  project. 

Job  set  1  consisted  of  a  series  of  one-tLme  individual  jobs  calling  proced¬ 
ures,  MCX3DIO  and  SRTTCOD,  which  were  developed  under  previous  efforts  as  generic 
M(DCDS  extract  and  sort/merge  interfaces  to  form  the  major  seven  unclassified 
process  subsets — the  1985  subset,  the  1986  subset,  the  five  arbitrary,  approxi¬ 
mately  equal  divisions  of  pre-1985  data — and  the  eighth,  classified  iMOCDS4  sub¬ 
set.  Division  into  subsets  allowed  processing  entirely  on  disk.  A  complete 
1985  s'obsec  was  formed  by  carbining  the  M00DS4  1985  subset  with  the  1985  sub¬ 
set  of  the  TGGA-funded  monthly  muni  MOODS  update  sat.  A  complete  1986  sucset 


was  extracted  when  the  cumulative  weekly  QUIPS  corrected  >iOCDS  set  frcm  the  TOGA 
grant  funded  MCODS-to-NCDD3  system  was  partitioned  to  start  1987. 

Job  set  2  consisted  of  the  series  of  steps  summarized  in  Table  3  using  the 
procedure  names  developed  during  this  effort. 

1)  SEL.  Partitioned  each  subset  into  a  salinity  subset  and  a  tenper- 
ature  subset. 

2)  SSL.  Processed  each  salinity  subset  to: 

a)  Correct  surface  salinity  tens  position  of  opt  based  on  valid 
salinity  reference  frcm  NiOODS  variable  header  or  frcm  the  Bauer- 
Robinson  Numerical  Atlas  climatological  value; 

B)  Correct  subsurface  tens  transition.  (Conversion  fran  FNOC  4D 
format  had  dubbed  the  value  3  into  the  tens  position  for  every 
level,  since  the  format  excluded  tens  position;  and 

C)  Reidentify  missing  levels  appropriately. 

3)  SW.  Processed  each  result  of  2)  to  remove  reference  salinity  frcm 
the  variable  header. 

4)  TTL.  Processed  each  temperature  subset  to  reidentify  missing  levels. 

5)  SZZ.  Processed  each  canbined  result  of  3)  and  4)  to  delate  reports 
with  entirely  missing  levels, 

6)  SMM.  Processed  each  result  of  5)  to  remove  missing  levels. 

Job  set  3  is  a  modified  version  of  the  NCDDS/QUIPS  NJ0B3  module.  Table  4 
provides  a  summary  of  the  sequential  job  set  3  process  steps.  Job  set  3  ap¬ 
plied  the  ship  code  garble  cross  reference  table  to  each  cruise  (CRUZ)  sequenced 
subset  using  program  RECLAS  to  correct  ship  codes.  The  development  of  the 
garble  cross  reference  table  was  the  major  effort  in  Task  3.  The  ship  call  sign 
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TABLE  3 


MCCDS4  MESSAGE  FILS  EDIT  PROCESS  JOB  SSI  2 


JOB 

STEP 

PRCCEDORE 

PROGRAM 

INITIAL 

FILE 

RSSLTLT 

FILE 

FUNCTION 

1 

?PAR,[N],SEL 

MPARSEL 

CLASSCN] 

CLSSSCN] 

CLTSSCN] 

Partition  the  file  'oetween 
temper  a  tue. ''salinity  subsets 

2 

PPAR, [N] ,3SL 
MPARSSL 

CLASStN] 

CLSSSCNlS 

Correct  tens  position  of 
salinity/old  missing  levels 

3 

PPAR,  [N]  ,SW 
MPARSW 

CI£SS[N]S 

CLSSSCNlV 

Remove  reference  salinity 
from  variable  header 

4 

SRT>’CCD 

SORIMRG 

CLSSS[N]V 

CLTSS[N] 

CLASS [N]V 

y^rge  files  together 

5 

PPAR,[M],TTL 

MPARm 

CLASSCN IV 

CLASSCNlT 

Reidentify  old  missing 

Tanperature  levels 

6 

PPAR, [N] ,SZZ 
MPARSZZ 

CLASS [NIT 

CLASSCNlZ 

Rercve  missing  profile 

Reports 

7 

PPAR,[N],SMM 

CLASSCNJZ 

CLASS [N]C 

Remove  missing  levels 

MPARSJ-IM 


Nota:  [N]  =  0(CLASS),l,2,3,4,5(pre~1985),85,86 
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TABLE  4 


MOODS 4  MESSAGE  FILE  EDIT  PROCESS  JOB  SET  3 


JOB 

STEP 

PROCSDJRE 

PROGRAM 

INPUT 

FILE 

RESULT 

FILE 

FjIJCTION 

1 

SRTMOOD 

CLASS [N]C 

TS-IPEN]  ,CY=A 

Cruise  sequence ( CRUZ ) 

2 

MOODIO 

TE;'IP[Nj  ,CY=A 

- 

Sunmary  print 

3 

MRECLAS 

SORIMRG 

XREFSHP 

SORIMRG 

XREFDLP 

RECIAS 

TEMPCMlC 

SHIPREFCFNGC) 

XRE?  TABLE 

GARBLE/UXA\'0/AS 

XREF  TABLE 

TEMP  [ M  ]  ,CY=A  'TEMP [N  ]  ,CY=B 

Garble  table  renaire  process 

4 

MOODIO 

TEMP[N] ,CY=B 

- 

Summary  print 

5 

MOODIO 

MOODEXT 

TEMP[N] ,CY=B 

TEMPEN]  ,cy=c 

Standard  weekly  garble 
correction  via  mcdify 

6 

SRIMOCD 

TS'IP[N]  ,CY=C 

TEMPEM] ,cy=d 

TLM/POS/CC  'SHI?  OC-EC ) 
sequence 

7 

NEGSPL 

TEMPCM] ,CY=D 

MSGACEM] 

TEMPEN]2,CY=A 

TEMPENJ3,CY=A 

Partition  file  into  s'lrface 
ARTST , CTEM/BATHY ( JJXX ) / 
TESAC(KX;<X) 

8 

SRTMOOD 

MSGACEN] 

TSMPEN] ,CY=A 

Count  reports 

9 

SRIMOOD 

•TEMPlN]2,CY=A 

TEMPEN]2,CY=B 

TLM/POS ( MGEO )  sequence 

10 

MOCDUPS 

TEMP[M]2,CY=B 

MSGHSVSEN] 

MSGHSVD(N) 

Identify  and  separace  dup¬ 
licate  report  sets  from 
singles 

12 

MOODIO 

MSGHSVDEN] 

- 

Print  duplicate  report  sets 

13 

SRI!-100D 

TEMP[N]3,CY=A 

TEMPEN]3,CY=B 

TLM/POS  (^'-GEO)  sequence 

14 

MOODUTS 

TEMP[N]3,C/=B 

msgbdsen] 

msgbdden] 

Identify  and  separate  dup¬ 
licate  report  sets  frcm 
singles 

15 

MOODIO 

MSGEDDEN] 

- 

Print  duplicate  report  sets 

field  in  the  historical  rressaga  sat  contains  various  processing  ancmalies  in¬ 
cluding  garbles  caused  by  radio  transmission  noise  that  resulted  in  extra,  lost 
and  perverted  characters;  FNCC  4D  format  changes,  e.g.,  area  of  interest  code 
had  been  prefixed  to  the  ship  call  sign  cn  11  May  1970;  truncated  ship  name 
s’ubstitution  when  the  call  sign  'rfas  absent  or  beyond  recovery;  substitution  of 
n'jmeric  pseudo-ship  code  and  subsequent  partial  loss  of  the  pseudo-ship  cross 
reference  table,  etc. 

The  garble  table  was  developed  using  a  special  version  of  prcgrain  MSGSPL 
chat  provided  rough  tables  of  proposed  name  reassignment  to  l<nown  valid  shio 
call  signs.  Each  entry  in  the  rough  tables  was  created  '^en  there  were  multiple 
reports  in  the  file  with  exactly  the  same  time  and  positions.  These  usually 
resulted  from  delayed  or  non  exact  duplicate  receipts  of  these  observations  on 
AUTCDI.N  or  reentry  of  a  report  at  FN’CC  more  than  72  hours  after  it  was  ta<en. 

The  rough  table  derived  frcm  each  major  subset  was  processed  independently  to 
ranove  duplicate  entries  while  retaining  a  count  of  the  number  of  duplicate  en¬ 
tries  as  a  measure  of  quality.  Objective  tools  were  developed  to  aid  in  the 
largely  S’ubjective  process  of  inter-  and  intra-table  comparison  to  further 
qualify  entries  for  a  final  garble  table.  An  objective  editor  called  EDIT,  a 
non-standard  CCC  product,  was  embedded  in  a  CCL  procedure  interface  to  facili¬ 
tate  ccmparisons.  Typically  a  particular  letter  combination  forming  part  of  a 
valid  ship  name  would  have  cn  the  order  of  50  cross  entries,  many  of  'which  would 
also  be  crosses  to  other  ship  codes  or  names.  As  a  result,  the  final  table  in¬ 
cluded  upwards  of  30  altf-.rnate  codes  for  a  single  platform.  Through  a  substan¬ 
tial  effort,  the  final  garble  cross-reference  augment  table  corrected  about  ten 
percent  of  the  ships  in  the  original  >500034  file  set  that  had  previously  been 


'onidentif led,  .\fter  applying  the  garble  table  call  sign  changes,  the  subsecs 
were  each  resaq^aenced  by  tbce/position/coiuntry/'shipcoda  OGEC)  for  input  to 
pr'ccran  >!SG3?L.  >’SGS?L  partitioned  each  subset  into  surface  .’'ISGAC  ( ARIGT/'CIZI'l ) , 
BAGHY  { JJ^CO  ,  and  TESAC  (I<KXX)  t%pe  subsets.  Processing  anded  at  this  point  for 
all  >!S'GAC  subsets.  These  MSGAC  reports  are  on  the  delivery  tapes  as  file  2. 

The  other  subsets  were  separately  resequencsd  by  tLxe/pcsiticn  (>!G£0)  for  each 
xajor  subset  and  individually  passed  to  program  MCCDuPS,  which  identified 
duplicate  pairs  (3ST3)  ar.d  and  partitioned  each  subset  into  a  singles  subset  and 
a  multioles  s’ubset. 


In  the  w-ee:<ly  MCCOS  correction  cycle  each  multiples  subset  requires  sub¬ 
jective  deletion  to  eliminate  duplicate  pairings  prior  to  the  NJ0B4  step,  but 
the  shear  vol'cme  of  duplicative  sets  in  this  historical  process  precluded  such 
subjective  processes.  Job  Set  3A,  Table  5,  "was  developed  and  applied  to  the 
five  pre-1985  s'ubsats  and  the  classified  subset.  It  should  be  noted  that  this 
process  was  not  developed  until  after  the  1985  and  1986  s’ubsets  were  shipped. 

Job  Set  3 A  replaced  the  usual  weakly  QUIPS  subjective  process  of  elLminating  non 
exact  duplicates  with  an  objective  process  based  on  experience  with  the  histor¬ 
ical  wee<ly  subjective  decisions.  Table  6  presents  the  simplistic  interpreta¬ 
tion  of  certain  MCCDS  field  elenent  values  used  in  this  objective  decision¬ 
making  process. 

Job  Sec  4,  Table  7,  is  a  modified  version  of  the  NJ0B4  module.  Job  Set  4 
reconbined  the  TESAC  (KKXX)  pieces  usi.ng  program  ^OJDKKX.  Then  program  MOCDLGO 
was  used  to  set  the  MCX3DS  quality  flags  in  the  CRUZ  and  >i3S0  sequences,  cctn- 
pleting  the  Jcb  Set  4  process  for  the  1985  and  1986  subsets.  For  other  subsets 


the  process  included  recombining  each  resultant  subset  with  its  corresponding 


TABLE  5 


MOODS 4  MESSAGE  FILE  EDIT  PROCESS  JOB  SET  3 A 


JOB  PROCEDURE  INPUT  RESULT 

STEP  PROGRAM  FILE  FILE  FUNCTION 


PPARSDP,[N],[TYPE]  MSG[TYPE][N]  MSG[ TYPE] [N]C  Perform  subjective 

MPARSDP  ^BG[■TYPE]  [N]D  emit  process  to  avoid 

dulicates  in  Job  Set  4 


Notes:  Job  Set  3A  was  not  applied  to  1985  and  1986  sets. 
N=0,l,2,3,4,5 
TYPE=HSVD,BDD 


TABLE  6 


Tests  a33'^.a 


NL’MBER 

OF 

LEVELS 


DEPTH  FLAG 


DEPTH  PARAM 
CODE 


PARAM  2  FLAG 


BITS  PER 

LEVEL 

(DEPTH) 

BITS  PER 
LEVEL 

(2ND  PARAM) 

FPB  (VARIABLE 
HEADER) 


MOODS  FIELDS  USED  TO  ELIMINATE  DUPLICATES 
sorted,  paired  cotiparison  of  MCX)DS  reports  of  the  same  type. 


In  the  historical  massage  file  the  shorter  profile  is 
presumed  to  'oe  the  result  of  objective  truncation  caused 
by  a  problem,  and  the  longer  trace  is  then  the  result  of 
subjective  operator  correction. 


Deselect  report  with  depth  flag  set. 


Deselect  more  precise  depth.  The  bul]<.  of  the  historical 
file  is  in  tens  of  feet.  One  report  at  a  precision  of 
feet  does  not  add  significance.  Not  valid  for  current 
real  time  file. 


Deselect  report  with  flag  set. 


A  measure  of  roughness.  The  roughest  profile  is  de¬ 
selected,  avoiding  the  large  depth  jumps  associated  with 
objective  truncation  process. 

A  measure  of  roughness.  The  roughest  profile  is  de¬ 
selected  tending  to  avoid  the  profile  with  spike,  except 
that  zero  bits  per  level  are  de-selected. 

De-select  report  with  more  reference  values.  Assumes 
that  lesser  cane  in  later  as  a  correction. 
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TABLE  7 

■!i| 
■  « 

MOODS 4  MESSASE  ? 

ILE  EDIT  PROCESS  JOB  SET  4 

JOB 

PROCEDURE 

INPUT 

RESULT 

STEP 

PROGRAM 

FILE 

FILE 

FUNCTION 

'k* 

1 

SRTMOOD 

MSGHSVDCNlC 

■TEMPA 

Resequence  l‘>SGED 

»;;h’ 

2 

MCCDKKX 

TEMPA 

TEMPB, TEMPO 

Recombine  TESAC  pieces 

V?i 

3 

SREMCOD 

TEMPB,TEMPC 

TEMPD 

Resequence  ship/tLme ( CRUZ ) 

v>' 

SORIMRG 

MSGBDD[N]C 

*'»! 

MSGBDS[N] 

CJ 

.vU 

4 

VODDIO 

TEMPO 

TEMPE 

>DODS  cruise  quality  process 

5 

SREMOCD 

TEMPE 

TEMPF 

Resequence  tLT.e/po3ition 

s 

(M3EO) 

!<.*• 

5 

MOODIO 

TEJ4PF 

TEMPG 

MOODS  geo  quality  process 

i 

7 

SRTMOOD 

TEMPG 

MOODS [N] FINAL 

Time/position  (>!GEO)  sequence 

'^'i 

MSGBDD[N]D 

MSGHS^7D[N]D 

i 

».*» 

( 

0 

Note;  Each  Job  4  Initiated 

via  procedure 

NJOB4ST/UTOGA  or  UJOB/AS 

m 

\i 

a 

0 

k 

' 

•S5i 

«•,** 

12 

:^i 

! 

: 

rt’l 
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subset  of  duplicate  reports  removed  in  the  Job  Set  3A.  Since  the  J0B3A  process 
was  necessarily  objective,  the  deleted  cbser'/aticns  -were  retained  so  that  future 
subjective  ccrrparisons  might  be  made. 

Job  Set  4A,  Table  8,  applied  the  newly  developed  program  MPARDEP  to  remove 
those  observations  exhibiting  non -increasing  depth  characteristics.  Those 
observations  were  ccft±>ined  to  form  file  3  of  the  shipment.  All  observations 
exhibiting  such  characteristics  predated  progren  S10>XX)D  and  are  related  to  the 
4D  format  archive. 

Job  Set  43,  Table  9,  combined  the  subsets  to  form  the  pre-1985  'unclassified 
and  classified  tapes. 

MCODS  £ditd:g  STATCS 

In  the  description  of  tasks  there  are  elem.ent3  that  were  not  done  by  this 
project  because  of  the  direction  N’AVOCEASO  took  in  impLmenting  >!CCBS.  In  Task 
1  no  comparisons  were  tmde  between  so’orce  sat  30  and  the  N'CCC  files  because 
NAVC)CEA.NO  had  decided  to  incl’ude  its  version  of  these  data  in  its  retrieval  sys¬ 
tem  and  deleted  them  from  MOCDS.  As  it  turned  out,  in  the  source  cede  15  data, 
cemprised  of  the  MCDC  3D1  file,  South  African  and  Japanese  data  incl'uded  in 
MCXODS  from  hydrocast  card  images,  the  redundant  WDC  SDl  portion  of  the  file  had 
already  bean  deleted  as  part  of  the  ^J0CDS3  update.  .As  described  previously,  the 
Japanese  part  has  been  sent  to  I4CDC.  NCCX3  has  arranged  to  obtain  a  complete 
'Update  from  South  Africa,  which  it  will  use  rather  than  the  older  NKXX)S  input 
file  to  update  its  USD2  file.  In  Tas.k  2  all  MBT's  for  which  CSI  could  calculate 
corrections  were  reviewed,  not  just  those  with  corrections  greater  than  .5’  C, 
and  the  corrections  were  applied  without,  regard  to  hanisphere.  This  resulted  in 
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TABLE  8 


M00DS4  MESS.A3B  FILE  EDIT  PROCESS  JOB  SET  4A 


JOB  PROCEDURE  I:'JPUT  RESULT 

STEP _ PROGRAM  FILE _ FILE _  FUNCIICN 


1 

PPARDEP 

MPARDEP 

MOODS [N] FINAL  MOODS [ N ] FINALGOOD 

>£)OD3[N]FINALBAD 

Ship/time 
sequence (CRUZ) 

2 

VODDIO 

MOODS [NJFINALBAD  - 

Reports  print 

TABLE  9 

MOOOS4  MESSAGE  FILL  EDIT 

PROCESS  JOB  SET  43 

JOB 

PROCEDURE 

INPUT 

RESULT 

STEP 

PROGR-AM 

FILE 

FILE 

FUNCTION 

1 

SRTMOCD 

MSGA:[N],N=1,85 

MAGT.!\PE1 

SCRIMRG 

MSGACO 

MAGTAPEA 

Conbine  sets  to 

MAGTAPEl 

MSGISACBTE 

tape 

2 

COPYBF 

MAGTAPE2 

MSGOACBTE 

Form  BT=E  set  on 
tape 

3 

SR'I>KX>D 

MOODS  [  N  ]  FIN  ALBAD ,  N= 

=1,5  MAGTAPE2 

SORTMRG 

MOODSOFINALSAD 

MAGTAPEB 

Ccmbine  sets  to 

MAGTAPE2 

MOODS15FIN ALBADBTE 

tape 

4 

COPYBF 

MAGTAPEB 

^JOODSO  FINALBADBTE 

Form  BT=E  set  on 
disk 

5 

SRTMtXD 

>ODDS [ N ] FINALGOOD , N=1 , 5  MAGTAPE3 ( FILEl 

) 

SORr-LRG 

MOODSOFINALG(XD 

MAGTAPEC (FILEl) 

Combine  sets  to 

MOODS16ACBTE 

MAGTAPE3(FILE2) 

tape 

6 

COPYBF 

MOODSOACBTE 

MAGTAPEC (FILE2) 

Move  BT=E  set  to 

MOODS 1 5 FIN ALBADBTE 

MAGTAPE3 (FILE3 ) 

cape 

7 

COPYBF 

moodsofinalbadbte 

MAGTAPEC (FILE3) 

Move  3T=E  set  to 

MAGTAPE3 

MA3TAPE4 

tape 

8 

CCPYBF(3) 

MAGTAPEC 

MAGTAPED 

Prepare  shipnvent 
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additional  corrections  to  some  of  the  Southern  Haciisphsre  reports  originally 
corrected  for  cur  Southern  Hemisphere  Atlas. 

In  Task  3  one  s'urface  only  reports  were  separated  out  of  the  MOCDS  file  into 
oheir  c^-/n  file,  and  CSI  recanrends  mintaining  them  separately  or  with  the  sur¬ 
face  marine  reports. 

The  edits  proposed  in  item  2  of  TasK  3  were  not  performed  because  the  tests 
have  evolved  into  a  portion  of  the  near -real  time  edit  sequence  that  requires 
interactive  decision  making.  Since  this  project  lacked  the  resources  to  select 
which  of  the  non  exact  duplicates  should  be  saved,  the  non  exacts  were  irerged 
'rack  into  the  file  after  MOCDUPS  identified  them  in  steps  10  and  14,  as  shown 
in  Taole  4,  with  a  setting  of  the  non  exact  duplicate  flag  based  on  the  cri¬ 
terion  in  JCB  3A.  The  subjective  decisions  required  will  be  sLmple  and  easy  to 
mei<a  once  QUIPS2  is  operational,  so  CSI  elected  not  to  use  this  project's  re¬ 
sources  on  the  problOT.  The  reports  with  depths  repeated  or  out  of  order  were 
separated  out  of  the  >ODDS  main  file  but  were  sent  as  file  3  on  the  update 
tapes.  These  reports  likewise  will  be  easy  to  edit  on  QUIPS2,  so  CSI  minimized 
its  efforts  on  them  for  this  project. 

The  IGOSS  tests  were  implemented  in  the  near -real  time  edit  sequence  in  the 
step  NCDDS/QuIPS  NJCB5.  This  step  was  not  applied  to  the  historical  data  set 
because  it  involved  extensive  subjective  'work.  Again,  QUIPS2  .may  mare  it  pos¬ 
sible  to  review  the  historical  traces  in  cruise  sets  but  that  is  be'/ond  the 
present  capability. 

In  this  project  CSI  did  not  modify  any  of  the  MCX5DS4  QC  tests,  so  that  these 
reports  can  be  merged  -with  the  other  source  sets  and  produce  a  consistent  QfZ 
level  throughout.  Addicionally,  since  NAVOCEANO  and  ^^RDA  are  improving  on 
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these  'QC  tests,  CSI  celieved  that  continued  effort  at  FNOC  in  parallel  was  not 
a  productive  use  of  the  project's  limited  resources. 

CSI  has  made  changes  zo  the  MOODDGO  and  MCCDEXT  programs  during  this  pro:ect 
which  have  been  submitted  to  the  FNCC  source  level  libraries.  The  only  signifi¬ 


cant  change  to  these  programs  that  is  not  documented  in  the  CSI-prepared  ?h;cc 


documentation  are  the  changes  to  the  variable  header  description  in  the  MOODS 
Data  Description.  That  2  page  change  was  part  of  this  project's  progress  reccrt 


~2  and  is  attached  to  this  report  before  the  references, 
to  prevent  MCCDLGO  and  .'-CCDEKT  from  setting  the  intrafile 


The  second  change  was 
duplicate  flag.  This 


duplicate  process  is  new  lone  MODDS /QUIPS  IWOBl  through  hJCE6. 


CCNCLCSICNS 

This  project  marked  the  first  time  that  IvCDC  directly  participated  in  the 
MCCCS  update  prccass.  The  results  shewed  there  are  data  in  MOODS  that  should 
be  in  the  .\CDC  files  and  an  initial  effort  was  made  to  transfer  those  data. 

N'ODC  is  using  the  MOODS  conversion  results  to  edit  the  USD2  file  and  is  adapt¬ 
ing  seme  of  the  MOODS  QC  testing  techniques.  The  result  is  that  the  USD2  file, 
although  the  newest  new  in  MOODS,  should  be  trodated  again  in  its  entirety  with¬ 
in  a  few  years. 

Assuming  NCDC  applies  the  MST  'TCS  corrections,  the  sane  applies  to  the  .'TST 
file  since  NCDC  changed  the  ship  names  to  NCDC  ship  codes  during  the  conversion 
to  UBT  and  has  eliminated  duplicates  in  this  file.  NODC  has  been  given  the 
XBT  source  code  5  file  and  will  be  inserting  it  in  UBT.  .^^ain,  when  that  pro¬ 
cess  is  carplete  the  entire  XBT  file  should  be  updated. 

This  participation  by  NODC  is  welccme  and  ccmnendable ,  and  CSI  hopes  that  it 
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will  continue  as  ic  is  clear  that  this  is  only  the  start  of  what  should  have 
been  happening  all  along. 

In  Septefrj>ar  1986  with  the  mini  MOODS  update  operating,  CSI  reached  a  long 
term  design  goal  of  MCCDS:  to  keep  an  up-to-date  classified  and  ’unclassified 
MOODS  stack  tape  in  the  FJ'.'OC  operational  center.  This  achievem.ent  was  not 
appreciated  by  either  rhiCC  nor  NAVOCEAMO,  and  it  may  be  several  years  h^efore 
that  event  is  repeated.  In  the  near  term  neither  the  Na'/y's  requirements  nor 
these  of  the  civilian  cofTtmunity  will  be  adequately  provided  with  complete  sets 
of  quality  controlled  data. 
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